first attested 1730s, but probably much older. The notion is the suit of clothes one was born in, i.e., no clothes at all. Compare Middle English mother naked "naked as the day one was born;" Middle Dutch moeder naect, German mutternackt.
in his birthday suit
biretta
Birmingham
Biro
birth
birthday
birthday suit
birthing
birth-mark
birthplace
birth-rate
birthright